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static  and  dynamic  images. 

.  The  role  of  the  human  observer  in  JPEG  compression  was  clarified.  The  human  observer’s 
visual  information  capacity  was  calculated  to  be  about  20  bits/min  ,  substantially  higher  than 
previous  estimates. 

.  New  formulas  were  developed  for  the  Fourier  transform  of  JPEG  basis  functions,  connecting 
JPEG  quantization  matrices  to  the  human  observer's  contrast  sensitivity  function. 

Crawford  masking  was  used  to  measure  the  visibility  of  lines  and  edges  following  abrupt  _ 
luminance  changes.  The  high  frame  rates  produced  higher  temporal  resolution  than  previous 
studies.  A  striking  asymmetry  between  light  and  dark  lines  was  found.  . 

b.  A  robust  test-pedestal  framework  was  developed  for  modeling  spatio-temporal  vision  with 
fewer  assumptions  than  previous  models.  In  this  framework  motion  processing  and 
hyperacuity  thresholds  are  directly  related  to  contrast  processing. 

c.  A  number  of  studies  on  motion  processing  developed  new  limits  on  the  human  visual 
system's  capabilities. 

d.  In  order  to  connect  psychophysics  results  to  underlying  physiological  mechanisms  new  • 
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Objectives  of  the  research  effort. 

a.  Apply  models  of  human  vision  to  several  areas  of  applied  image  processing  (image  compression  and 
image  quality),  such  as  high  definition  television  and  teleconferencing. 

b.  Develop  robust  "ideal-observer"  methods  for  modeling  spatio-temporal  vision  with  fewer  assumptions 
than  previous  models.  The  test-pedestal  framework  for  spatio-temporal  interactions  uses  the  same  test 
pattern  with  multiple  pedestals  in  different  phase  relationships  to  learn  about  the  properties  of  the 
underlying  mechanisms.  We  seek  to  connect  motion  processing  to  contrast  processing. 

c.  Discover  the  limits  of  spatial  processing  while  the  stimuli  are  rapidly  moving,  flickering  or  jittering. 
Develop  models  to  account  for  this  data. 

d.  Connect  our  psychophysics  research  to  the  underlying  physiological  mechanisms.  In  this  regard  we 
have  found  that  new  techniques  are  needed  to  learn  about  nonlinear  visual  processing  so  we  are 
improving  methods  for  the  nonlinear  analysis  and  source  localization  of  visual  evoked  potentials  and 
other  biopotentials. 


Overview  of  this  final  report. 

The  past  three  years  have  been  very  productive  for  our  research  group.  Based  on  AFOSR  support 
we  have  21  papers  either  published  or  submitted  for  publication  (excluding  research  supported  solely  by 
NIH).  I  am  including  with  this  report  a  copy  of  those  papers  that  were  not  previously  sent  (or  those  for 
which  preliminary  versions  were  sent).  In  this  report  I  will  refer  to  some  of  these  papers.  I  will  use  the 
notation  "#4"  to  mean  paper  number  4  in  the  list  of  references  at  the  end  of  this  report.  If  I  were  to 
summarize  the  work  done  on  each  paper  this  document  would  become  excessively  lengthy.  Instead,  this 
summary  will  provide  a  short  summary  of  our  AFOSR  supported  activities  in  the  following  four  areas:  1) 
A  brief  summary  of  our  research  on  connecting  the  insights  from  human  vision  to  applied  topics  (image 
compression  and  image  quality).  2)  A  summary  of  our  work  on  vernier  acuity  to  show  our  approach  to 
modeling.  An  outgrowth  of  this  research  is  our  collaboration  with  other  Berkeley  faculty  to  develop  a  user- 
friendly  environment  for  modeling  vision.  3)  A  brief  summary  of  our  research  on  elaborating  the 
mechanisms  of  motion  processing.  4)  A  summary  of  our  work  measuring  biopotentials.  In  order  to  connect 
our  psychophysical  data  and  models  to  underlying  physiological  mechanisms  we  have  executed  several 
visual  evoked  potential  studies  and  have  developed  new  methodologies  for  studying  nonlinear  physiological 
processing. 

The  full  details  for  much  of  our  AFOSR  supported  research  can  be  found  in  the  published  articles 
that  are  either  enclosed  with  this  report  or  have  been  sent  previously.  Some  of  this  completed  research  will 
be  reviewed  briefly  in  this  report.  Abstracts  of  some  of  our  unpublished  AFOSR  supported  work  will  be 
included  in  this  final  report. 


1)  Image  compression  and  image  quality.  Eight  of  the  papers  that  have  been  written  during  this 
grant  period  are  connected  with  image  compression  and  the  assessment  of  image  quality  [#1,  #2,  #4,  #5, 
#7,  #1 1,  #14,  #18].  The  field  of  image  compression  is  growing  very  rapidly  because  of  the  arrival  of 
HDTV,  teleconferencing  and  picture  phone  and  also  because  of  developing  standards  by  committees  such 
as  JPEG  and  MPEG.  The  standards  developed  by  JPEG  and  MPEG  are  ideal  for  the  vision  research 
community.  The  quantization  algorithm  is  based  squarely  on  properties  of  the  human  visual  system.  Much 
more  information  from  researchers  in  human  vision  is  needed  on  how  to  do  context  dependent  image 
compression  and  image  quality  evaluation.  That  is,  different  types  of  image  degradation  will  be  more  or  less 
visible  depending  on  the  local  context  of  the  image.  This  is  the  topic  of  visual  masking.  One  of  the 
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directions  of  both  our  basic  research  and  our  applied  research  is  to  show  that  the  calculation  of  masking 
magnitude  is  more  difficult  than  commonly  believed.  We  are  able  to  demonstrate  situations  with  strong 
pedestals  in  both  space  and  time  where  the  masking  is  minimal.  Based  on  these  results  we  have  calculated 
that  the  human  visual  system  is  able  to  take  in  much  more  information  than  is  transmitted  by  normal 
compression  schemes.  We  believe  that  our  research  is  relevant  whenever  the  goal  is  to  achieve  perceptually 
lossless  compression.  The  best  summary  of  our  research  for  static  images  is  our  recent  SPIE  paper  [#18]. 

One  area  of  special  interest  involves  temporal  masking.  We  are  interested  in  how  much  compression 
is  possible  when  there  is  a  sudden  change  in  scenes.  When  a  new  scene  appears  in  an  image  sequence  a 
tremendous  amount  of  new  information  is  needed  to  represent  the  scene.  We  want  to  learn  what  aspects  of 
the  sudden  scene  change  are  not  visible  to  the  human  observer  immediately.  Amnon  Silverstein  and 
Qingmin  Hu  will  be  doing  their  doctoral  dissertations  in  this  area.  The  following  abstract  shows  one  of  our 
directions  in  his  research. 

LINE  DETECTION  UNDER  TEMPORAL  MASKING  (ARVO  1992) 

The  Crawford  masking  flash-on-flash  paradigm  was  employed  to  study  the  effect  of  temporal 
masking  on  the  detection  of  a  thin  line.  We  wanted  to  use  finer  temporal  resolution  (5  msec)  than 
previous  studies.  The  adapting  background  was  90  cd/m2.  The  masker  was  a  uniform  square 
field  of  6  degrees  with  a  luminance  increment  of  50  cd/m2.  The  masking  field  was  flashed  on  for 
500  msec.  The  target  was  a  2  minute  wide  vertical  line  which  was  briefly  (5  msec)  flashed  on  at 
the  center  of  the  masking  field.  Detection  thresholds  were  measured  at  different  SOAs  of  the 
masking  flash  with  a  sampling  interval  of  5  msec.  Data  were  obtained  by  the  method  of  constant 
stimuli  with  multiple  responses.  Three  major  effects  were  observed:  (l)The  masking  of  a  thin  line 
by  a  uniform  field  had  a  very  short  time  course:  about  25  msec  full  width  at  half  maximum.  (2)The 
psychometric  function  for  the  thin  line  detection  under  temporal  masking  was  steeper  than  the  no- 
masking  condition.  The  exponent  of  the  psychometric  (transducer)  function  relating  d'  to  contrast 
was  2.49±0.25  and  1.79±0.16  for  the  mask  and  no-mask  conditions.  (3)  Crawford  masking 
functions  were  obtained  for  a  2  min  line  and  a  4  min  line.  The  detection  thresholds  of  the  2  min 
line  were  almost  double  the  4  min  line  thresholds  in  the  no-mask  case  and  only  about  10%  higher 
with  the  masker.  This  is  consistent  with  the  finding  that  high  spatial  frequency  detectors  do  not 
respond  well  to  the  luminance  transient.  (4)  One  of  our  most  interesting  findings  is  the  strong 
asymmetry  between  light  and  dark  test  lines,  indicating  temporal  delays  in  the  gain  control 
mechanism. 

Another  interesting  result  in  our  work  relevant  to  image  coding  is  the  paper  by  Klein  &  Beutter 
(1992,  reference  #7)  because  it  has  caused  a  stir  in  several  fields.  In  image  compression  (and  image 
processing  in  general)  one  wants  to  use  filters  that  are  localized  in  both  space  and  spatial  frequency.  Gabor 
once  made  a  claim  that  for  real-valued  functions,  the  Hermite  functions  (Hermite  polynomials  times  a 
Gaussian)  minimize  the  joint  space-spatial  frequency  uncertainty.  What  we  showed  was  that  for  the  class  of 
functions  that  are  an  mth  order  polynomial  times  a  Gaussian,  the  Hermite  functions  maximize  the  joint 
uncertainty.  Individuals  from  different  disciplines  have  indicated  the  usefulness  of  this  result  (e.g.  I  have 
been  informed  that  Hermite  functions  characterize  the  profile  of  laser  beams  and  based  on  Gabor's  work  it 
had  been  thought  (before  our  paper)  that  this  was  a  good  profile). 

The  applied  vision  field  of  image  compression  and  image  quality  is  moving  extremely  rapidly  in  the 
engineering  community.  It  is  simply  shocking  to  me  to  see  how  little  research  in  this  area  is  being  done  by 
vision  researchers,  who  have  some  understanding  of  underlying  mechanisms.  We  intend  to  submit  a 
proposal  to  AFOSR  to  continue  the  work  we  have  started  in  this  field. 


2)  Vernier  acuity  and  modeling.  Using  the  test-pedestal  paradigm,  we  initially  established  a 
connection  between  vernier  acuity  and  contrast  discrimination  using  sinusoidal  stimuli  (Hu,  Klein  & 
Carney,  1992,  reference  #19).  Rather  than  using  a  complex  model  with  many  assumptions  we  showed  that 
to  first  order,  vernier  acuity  was  well  predicted  by  contrast  discrimination  thresholds  when  both  tasks  are 
expressed  in  the  same  contrast  units.  These  results  can  be  explained  by  both  tasks  using  common 
underlying  mechanisms.  However  several  notable  exceptions  were  evident.  At  high  spatial  frequencies 
vernier  thresholds  degraded  faster  than  expected  when  compared  to  contrast  discrimination  data.  Moreover, 
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the  tvi  slope  was  always  shallower  for  the  vernier  task.  We  have  been  exploring  other  stimulus 
configurations  including  different  line  lengths  and  central  gaps  in  both  tasks  to  determine  the  source  of  the 
deviations  from  predictions  based  on  contrast  discrimination.  At  high  spatial  frequencies,  decreasing  grating 
length  hurt  contrast  discrimination  thresholds  but  actually  improved  vernier  acuity.  The  vernier  task  is 
presumably  performed  by  oriented  mechanisms,  the  long  grating  possibly  diluted  or  smeared  out  the 
orientation  cue  which  reduced  the  effectiveness  of  oriented  mechanisms.  The  difference  in  tvi  slope  is  likely 
due  to  the  use  of  oriented  mechanisms.  If  vernier  acuity  involves  mechanisms  oriented  away  from  the 
pedestal  orientation  it  would  avoid  some  of  the  contrast  masking  effects  observed  for  the  contrast 
discrimination  task,  thereby  resulting  in  a  different  tvi  slope.  Further  details  on  this  research  is  provided  in 
the  following  two  ARVO  abstracts. 

COMPARISON  OF  GRATING  VERNIER  ACUITY  AND  CONTRAST 
DISCRIMINATION  (ARVO  1990) 

A  test-pedestal  approach,  in  which  a  test  grating  was  superimposed  on  a  masking  pedestal,  was 
used  to  compare  sinusoidal  grating  vernier  acuity  and  contrast  discrimination  thresholds.  In  the 
contrast  discrimination  task,  subjects  were  asked  to  detect  contrast  increments  (AC  cosfx)  in  the 
presence  of  a  base  contrast  (C  cosfx).  In  the  vernier  task,  a  test  grating  (AC  sin(fx+(J>/2))  was 
added  to  one  half  of  the  pedestal  grating  (C  cosfx)  to  produce  a  vernier  offset  (8).  The  relationship 
between  contrast  increment  (AC)  and  vernier  offset  (8)  is  given  by:  AC  =  2C  sin  (<J>/2)  in  which 
phase  shift  (<]))  equals  2ttfd.  Thresholds  (d'=l)  were  determined  by  a  self-paced  method  of 
constant  stimuli  with  multiple  responses.  For  contrasts  below  10  times  detection  threshold,  Rvc  > 

1  across  all  spatial  frequencies  (where  RVc  is  the  ratio  of  the  vernier  threshold  to  the  contrast 
discrimination  threshold  expressed  in  %  contrast,  AC),  and  Rvc  increased  with  increasing  spatial 
frequency.  At  a  spatial  frequency  of  2  c/d,  there  was  a  cross-over  between  vernier  and  contrast 
discrimination  thresholds  as  a  function  of  pedestal  contrast:  at  contrasts  below  about  10  times 
detection  thresholds,  Rvc  ~  2,  for  contrasts  above  that,  RvC  =  0.5.  At  high  spatial  frequency  (20 
c/d),  Rvc  is  about  10  across  all  pedestal  contrasts.  At  spatial  frequencies  between  5  to  10  c/d,  for 
contrast  above  10  times  detection  threshold,  RVc  ®1.  Thus,  under  these  conditions,  contrast 
discrimination  predicts  grating  vernier  acuity.  A  hypothesis  that  the  same  mechanism  is  involved 
in  performing  these  two  tasks  at  intermediate  spatial  frequencies  will  be  discussed. 


PREDICTING  GRATING  VERNIER  ACUITY  FROM  CONTRAST 
DISCRIMINATION:  THE  EFFECT  OF  GRATING  LENGTH  (ARVO  1991) 

Last  year,  we  used  a  test-pedestal  approach,  in  which  a  test  grating  was  superimposed  on  a 
masking  pedestal,  to  compare  sinusoidal  grating  vernier  acuity  (in  contrast  units)  and  contrast 
discrimination  thresholds.  We  found  that  grating  contrast  discrimination  thresholds  are  in  good 
agreement  (within  30%)  with  vernier  acuity  at  spatial  frequencies  between  5  and  10  c/deg. 
However,  at  high  spatial  frequencies  (20  c/degree),  vernier  thresholds  are  about  10  times  that  of 
contrast  discrimination  at  all  pedestal  contrasts.  We  wonder  whether  shortening  the  length  of  the 
grating  may  increase  the  relative  effectiveness  of  the  optimal  mechanism  to  perform  the  vernier  task 
at  high  spatial  frequencies.  To  test  this  hypothesis,  the  same  approach  was  used  to  measure  the 
grating  vernier  and  contrast  discrimination  thresholds  at  various  grating  lengths  at  a  pedestal 
contrast  of  nine  times  detection  threshold  at  spatial  frequencies  of  10  and  20  c/deg.  At  10  c/deg, 
vernier  threshold  increases  as  grating  length  is  shortened,  similar  to  contrast  discrimination.  But  at 
20  c/deg,  vernier  threshold  does  not  vary  much  with  the  grating  length  while  contrast 
discrimination  thresholds  increase  with  decreasing  grating  length.  Thus,  when  thresholds  are 
plotted  as  a  function  of  grating  length,  contrast  discrimination  and  vernier  functions  intersect  at  a 
grating  length  of  2  minutes  for  20  c/deg  gratings. 

We  are  now  also  comparing  vernier  acuity  and  grating  detection  thresholds  in  the  presence  of 
oriented  masking  gratings  [17].  The  masks  have  large  effects  at  orientations  somewhat  different  from  the 
vernier  target  grating.  This  masking  effect  is  greatly  reduced  in  the  grating  detection  task.  In  summary, 
vernier  acuity  for  the  most  part  is  well  predicted  by  contrast  discrimination.  Future  models  of  vernier  acuity 
will  have  to  consider  properties  of  later  stages  of  processing  in  order  to  account  for  performance  where 
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contrast  discrimination  does  not  predict  vernier  acuity.  Such  models  will  likely  include  multiple  mechanisms 
at  different  spadal  scales,  orientations,  and  densities. 

We  have  spent  some  time  working  with  various  modeling  tools,  such  as  the  early  vision  emulation 
software  (EVE),  Mathematica  and  Matlab,  but  have  found  each  lacking  in  ease  of  use  or  flexibility.  Our 
frustration  with  these  tools  lead  to  regular  meetings  with  several  other  UCBerkeley  faculty  members 
discussing  the  requirements  for  a  powerful  vision  modeling  environment.  A  group  of  us  (Malik,  Banks, 
Schor,  Feldman,  Treisman,  and  Klein)  have  recently  submitted  a  computational/biological  vision  proposal 
to  AFOSR.  One  aspect  of  this  proposal  is  to  develop  a  user-friendly  modeling  environment  that  could  be 
used  by  the  larger  vision  community.  Such  a  tool  would  prove  invaluable  in  extending  our  understanding  of 
spatial  and  temporal  vision. 

3)  Mechanisms  involved  in  motion  processing. 

We  have  carried  out  a  number  of  studies  that  are  designed  to  clarify  the  mechanisms  that  underlie 
motion  perception.  Thom  Carney  has  written  up  several  of  these  studies  [6,  12,  20]  and  they  are  included 
with  this  report,  so  they  will  not  be  discussed  further.  The  manuscripts  for  two  further  studies  that  were 
reported  by  Amnon  Silverstein  at  ARVO  are  nearly  completed.  The  first  study  examines  the  mechanisms 
underlying  our  ability  to  make  good  vernier  judgments  for  moving  stimuli  [22].  We  provide  evidence  that 
the  task  is  accomplished  by  oriented  mechanisms.  Also  we  are  developing  a  simple  model  that  is  able  to 
account  for  the  data.  A  brief  summary  of  this  research  is  provided  by  the  following  ARVO  abstract: 

VERNIER  ACUITY  DURING  IMAGE  ROTATION  AND  TRANSLATION  (ARVO,  1990) 

It  has  been  proposed  that  vernier  acuity  thresholds  are  achieved  by  means  of  an  orientation  cue. 
Since  the  orientation  information  is  constant  during  translational  motion,  this  model  is  consistent 
with  the  remarkable  fact  that  vernier  acuity  is  unaffected  by  image  motion  at  velocities  less  than 
2.5°/s.  (Westheimer  and  McKee  1975).  However,  such  a  mechanism  would  be  strongly  affected 
by  rapid  changes  in  orientation.  Using  a  three  dot  vernier  acuity  task  with  a  0.2  s  presentation,  we 
have  assessed  performance  during  image  translation  and  rotation.  In  addition  to  velocity,  we 
varied  luminance  and  the  separation  between  the  dots.  We  found  that,  while  low  luminance 
elevated  thresholds  to  linear  targets,  luminance  has  relatively  little  effect  on  thresholds  to  rotatory 
targets.  Thresholds  to  linearly  translating  targets  were  compared  with  rotating  targets  of  equal 
tangential  velocity.  As  expected,  linear  translational  motion  did  not  elevate  thresholds.  However, 
rotational  motion  quickly  degraded  vernier  acuity.  Thresholds  became  elevated  with  as  little  as 
0.3°/s  tangential  velocity,  an  order  of  magnitude  slower  than  the  linear  case.  These  facts  are 
consistent  with  a  model  of  vernier  acuity  which  emphasizes  the  use  of  orientation  cues. 

Westheimer,  G.  and  McKee,  S.P.  (1975)  Visual  acuity  in  the  presence  of  retinal  image  motion.  J. 

Opt.  Soc.  Am.  65  847-850 


The  second  study  [23]  examines  the  rate  at  which  relative  motion  can  be  judged.  This  is  c’osely 
related  to  the  question  of  what  are  the  temporal  constraints  on  the  "binding  problem".  Suppose  an  object 
consisting  of  several  dots  is  vibrating  rapidly  in  sinusoidal  oscillation.  We  are  interested  in  what  is  the 
highest  temporal  frequency  for  which  the  object  can  be  perceived  as  rigid.  These  results  will  provide 
constraints  on  models  of  object  recognition.  The  following  ARVO  abstract  summarizes  this  research. 

RELATIVE  MOTION  DISCRIMINATION  (ARVO,  1991) 

Motion  of  very  small  amplitudes  (5  sec  or  less)  can  be  detected  when  the  frequency  of  motion 
is  optimal  (2-4  Hz.)  (Tyler  and  Torres  1972).  Previous  research  has  shown  that  under  some 
conditions,  relative  motion  can  be  discriminated  whenever  motion  can  be  detected  at  all  (Mowafy, 
Lappin,  Anderson,  and  Mauk  1991).  In  this  experiment,  we  determined  the  range  of  frequencies 
over  which  this  relationship  holds.  We  used  two  sinusoidally  oscillating  dots  to  determine  the 
temporal  frequency  response  function  of  relative  motion  discrimination.  The  subject  was  presented 
with  two  dots  that  either  oscillated  inphase  (the  dots  moved  in  the  same  direction)  or  in 
counterphase  (the  dots  moved  in  opposite  directions).  The  subject  was  required  to  discriminate 
between  the  two  types  of  motion  for  a  range  of  temporal  frequencies.  When  the  temporal  frequency 
of  the  presented  stimuli  is  optimal  (~4  Hz),  the  visual  system  is  remarkably  adept  at  discriminating 
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relative  motion.  However,  a  temporal  frequency  of  as  low  as  8  Hz  degrades  thresholds  and  12  Hz 
increases  them  by  a  factor  of  5.33  ±  1.33. 

An  important  new  development  for  my  group  is  that  Tina  Beard  has  just  joined  us.  We  look  forward 
to  many  years  of  fruitful  collaboration.  This  collaboration  began  this  past  year  while  she  was  in  Houston 
with  Dennis  Levi  and  we  worked  on  the  following  project  together.  We  were  interested  in  the  dynamics  of 
the  localization  process  as  is  described  in  the  following  abstract: 

SPATIOTEMPORAL  PROPERTIES  OF  LOCALIZATION  MECHANISMS  (ARVO,  1992) 

Localization  judgments  for  two  abutting  lines  are  thought  to  reflect  the  responses  of  orientation 
tuned  filters  sensitive  to  both  stimuli;  however,  when  the  two  lines  are  widely  separated  in  space, 
position  is  thought  to  be  coded  by  local  sign  mechanisms.  The  temporal  dependence  between 
reference  and  test  lines  is  not  known.  We  measured  localization  thresholds  across  various 
interstimulus  intervals  (ISIs)  for  abutting  and  spatially  separated  reference  and  test  lines  at  set 
contrasts  above  detection  threshold.  The  lines  were  presented  on  a  uniform  field  with  a  mean 
luminance  of  132  cd/m2,  for  a  duration  of  150  msec.  Observers  discriminated  the  direction  and 
amount  of  offset  between  the  reference  and  test  lines  using  a  5  alternative  rating  scale.  As  expected 
based  on  Weber's  Law,  threshold  increased  proportionally  with  the  spatial  separation  of  the  two 
simultaneously  presented  lines.  For  abutting  lines,  thresholds  increased  dramatically  with  even  a 
very  brief  (3.5  msec)  ISI,  perhaps  due  to  masking  (Westheimer  and  Hauske,  1975).  For  wide 
spatial  separations  (e.g.,  90  min),  localization  thresholds  were  independent  of  ISI.  We  can  rule  out 
edge  artifacts  or  unwanted  position  cues  since  thresholds  for  widely  separated  targets  were 
independent  of  viewing  distance.  Our  results  are  consistent  with  the  notion  that  the  precision  of 
spatial  localization  is  constrained  by  different  mechanisms  when  the  target  lines  are  abutting,  and 
when  they  are  well  separated.  The  most  precise  localization  thresholds  occur  when  the  target  lines 
are  presented  in  close  spatial  and  temporal  proximity  as  might  be  expected  from  the  response  of  a 
single  filter  sensitive  to  the  contrast  of  both  lines;  for  widely  separated  targets,  thresholds  are 
virtually  independent  of  ISI,  consistent  with  a  local  sign  mechanism  in  which  the  two  lines  are 
processed  by  separate  filters,  and  their  position  labels  compared. 

We  believe  that  models  of  motion  processing  are  still  in  a  very  immature  stage.  Most  present  models 
are  satisfied  with  qualitative  agreement  with  data  rather  than  quantitative  agreement.  We  intend  to  improve 
on  these  models  over  the  next  few  years. 

4)  Connecting  psychophysics  to  physiology:  The  visual  evoked  potential. 

One  of  our  goals  is  to  connect  our  psychophysics  experiments  to  physiology.  My  graduate  student 
Heidi  Baseler  is  doing  her  doctoral  disseration  on  Visual  Evoked  Potentials  (VEP).  Although  she  is  not 
being  supported  by  AFOSR,  I  consider  a  small  amount  of  time  I  have  spent  on  this  project  to  be  AFOSR 
supported  and  we  are  citing  AFOSR  support  in  all  of  our  publications.  One  of  our  goals  is  to  use  the  stimuli 
of  our  AFOSR  psychophysics  research  in  our  VEP  studies. 

4a)  The  test-pedestal  approach.  The  power  of  the  test-pedestal  approach  for  revealing  underlying 
mechanisms  in  our  psychophysical  studies  led  us  to  record  visual  evoked  potentials  (EP)  to  similar  stimuli 
with  the  goal  of  cortical  functional  localization.  The  temporally  varying  test  pattern  was  identical  across 
conditions,  the  different  static  pedestal  patterns  determined  the  stimulus  condition:  dynamic  vernier,  motion, 
contrast  modulation  or  counterphase  sinewave.  This  approach  has  the  advantage  that  the  test  signal  that 
generates  the  EP  was  identical  across  conditions.  The  results  were  compatible  with  psychophysical  results 
described  above  but  functional  localization  was  not  possible  with  the  limited  number  of  recording  channels 
available  to  us.  A  summary  of  this  work  is  contained  in  the  following  ARVO  abstract. 

A  TEST-PEDESTAL  PARADIGM  FOR  USE  WITH  VISUAL  EVOKED  POTENTIALS 
(ARVO,  1990) 

The  visual  evoked  potential  (EP)  to  a  temporally  modulated  test  grating  was  evaluated  in  the 
presence  of  a  variety  of  static  pedestal  gratings.  The  test  pattern  was  always:  AC  cos(fy)*cos(cot). 

The  pedestal  gratings  were  chosen  such  that  the  phenomenal  appearance  of  the  stimuli  included  a 
moving  vernier,  a  jittering  grating,  two  types  of  contrast  modulation  and  a  counterphase  grating. 
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The  stimulus  field  was  divided  into  left  and  right  halves  which  contained  the  horizontal  gratings. 
The  pedestal  gratings  utilized  to  generate  the  stimuli  described  include: 


Left  Half 

Right  Half 

Appearance 

Blank 

(C+AC)  cosfy  +  test 

Contrast  Modulation. 

C  cosfy 

(C+AC)  cosfy  +  test 

Contrast  Modulation,  appearance-disappearance 

Blank 

(C+AC)  sinfy  +  test 

Up  and  down  motion  of  sinewave. 

C  cosfy 

(C+AC)  sinfy  +  test 

Dynamic  vernier,  appearance-disappearance. 

Blank 

Blank  +  test 

Counterphased  sinewave. 

This  approach  has  the  advantage  that  the  temporally  modulated  signal  generating  the  EP  is  constant 
across  conditions.  Any  condition  dependent  changes  in  the  EP  results  from  pedestal  difference 
which  don't  by  themselves  produce  an  EP.  This  provides  a  convenient  means  for  comparing 
diverse  perceptual  tasks  as  well  as  determining  the  effects  of  pedestal  intensity  on  the  EP  to  a 
common  test  pattern.  The  relative  strengths  of  the  fundamental  frequency  and  the  second  harmonic 
components  of  the  EP  were  compared  across  tasks.  In  the  presence  of  pedestal,  the  overall 
amplitude  of  the  EP  at  the  second  harmonic  decreased  for  all  conditions.  The  masking  as  a  function 
of  pedestal  strength  compare  well  with  psychophysical  results  on  these  types  of  tasks. 


4b)  Topography  of  diverse  stimuli.  More  recently  we  teamed  up  with  Anthony  Norcia  and  Peter 
Wong  (ARVO  1991)  who  had  collected  EPs  to  a  variety  of  stimulus  categories,  vernier  jitter,  color, 
onset/offset,  and  counterphase  patterns  using  a  21  electrode  recording  array.  Our  analysis  of  the  data 
revealed  some  cortical  specificity  for  the  type  of  stimulus  pattern.  Unfortunately,  the  stimuli  were  not 
selected  according  to  the  test-pedestal  paradigm  which  complicated  comparisons  across  stimulus  conditions. 
A  summary  of  this  research  is  presented  in  the  following  ARVO  abstract: 


VEP  TOPOGRAPHY  OF  PERCEPTUALLY  DIVERSE  STIMULI  (ARVO  1991) 

Technological  advances  in  computer  mapping  programs  and  statistical  analysis  methods  have 
made  VEPs  an  increasingly  sensitive  and  convenient  technique  for  localizing  cortical  function.  By 
selecting  visual  stimuli  that  vary  along  distinct  perceptual  dimensions,  we  can  compare  the  scalp 
topography  of  such  visually  diverse  phenomena  as  pattern  onset/offset,  counterphase,  vernier, 
jitter,  temporal  frequency  and  color.  Ten  different  steady-state  stimuli  were  presented  to  14  normal 
subjects  whose  VEPs  were  recorded  from  21  electrode  sites.  In  all  cases,  the  stimuli  were 
vertically  oriented,  2  c/deg  square  wave  gratings,  undergoing  square  wave  temporal  modulation. 
Stimuli  varied  as  follows:  pattern  onset/offset  and  counterphase  of  luminance  and  isoluminant 
color  gratings,  counterphase  and  jitter  at  3  and  10  Hz,  symmetric  dynamic  vernier  (90°  phase  shift 
in  both  directions)  and  asymmetric  dynamic  vernier  (180°  phase  shift  in  one  direction).  Condition- 
specific  differences  were  found  in  signal  magnitude  and  distribution.  Fourier  analysis  revealed  a 
strong  response  at  all  even  harmonics  to  the  jitter,  counterphase  and  symmetric  vernier  stimuli, 
whereas  the  onset/offset  and  asymmetric  vernier  stimuli  had  peaks  at  both  odd  and  even 
harmonics.  Occipital  sites  showed  the  greatest  activity  in  all  conditions,  as  expected  from  a  visual 
task.  Of  greater  interest  was  the  relative  differences  in  activity  in  the  temporal,  parietal  and  central 
electrodes  between  conditions.  In  particular,  the  pattern  onset/offset  and  asymmetric  vernier 
stimuli  generated  significant  activity  in  the  central  electrodes  relative  to  the  counterphase  and 
symmetric  vernier  conditions.  Further  conditional  differences  were  found  and  will  be  discussed  in 
more  detail. 

Articles:  Published  and  In  Press  (January  1,  1989  -  July  31,  1992). 
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1.  Klein,  S.  A.  (1990).  High  resolution  and  image  compression  using  the  discrete  cosine  transform. 
SPIE:  Human  Vision  and  Electronic  Imaging,  Methods  and  Application.  1249,  135-146. 
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for  Informational  Display  90  Digest  21, 456-459. 

3.  Klein,  S.  A.,  Casson,  E,  and  Carney,  T.  (1990).  Vernier  acuity  as  line  and  dipole  detection.  Vision 
Res.  30,  1703-1719. 

4.  Klein,  S.  A.  &  Carney,  T.  (1991).  "Perfect"  displays  and  "perfect"  image  compression  in  space  and 
time.  Human  Vision,  Visual  Processing  and  Digital  Display  II.  Bernice  E.  Rogowitz,  Jan  P.  Allebach 
and  Michael  Brill,  Editors,  Proc.  SPIE  1453, 190  -  204. 

5.  Klein,  S.  A.  &  Carney,  T.  (1991).  How  the  number  of  required  gray  levels  depends  on  the  gamma  of 
the  display.  Society  for  Informational  Display  91  Digest  22, 623-626. 

6.  Carney,  T.  &  Shadlen,  M.  N.  (1992).  Binocularity  of  early  motion  mechanisms:  comments  on 
Georgeson  and  Shackleton.  Vision  Research  32,187-191. 

7.  Klein,  S.  A.  &  Beutter,  B.  (1992).  Minimizing  and  maximizing  the  joint  space-spatial  frequency 
uncertainty  of  Gabor-like  functions.  J.  Opt.  Soc.  Am.  A.  9,  337-340. 

8.  Klein,  S.  A.  (1992).  Optimizing  the  estimation  of  nonlinear  kernels.  Nonlinear  Vision.  Eds.  Robert 
B.  Pinter  and  Bahrain  Nabet.  CRC  Press.  109  -  170. 

9.  Klein,  S.  A.  (1992).  Channels:  Bandwidth,  channel  independence,  detection  vs.  discrimination. 
Chapter  in  Channels  in  the  visual  nervous  system:  Neurophysiology,  psychophysics  and  models. 
Blum,  ed.  11-27. 

10.  Klein,  S.  A.  (1992).  An  EXCEL  macro  for  averaging  data.  Behavior  Research  Methods, 
Instruments,  &  Computers  24,  90-96. 

11.  Hu,  Q.,  Klein,  S.  A.,  &  Carney,  T.  (1992).  Temporal  luminance  masking  in  the  coding  of  video 
signals.  Society  for  Informational  Display  92  Digest  23,  255  -  258. 

12.  Carney,  T.,  Schor,  C„  Steinman,  S.  &  Wilson,  N.  (1992).  Assessment  of  binocular  integration  in 

amblyopia  using  a  motion  stimulus.  In  Noninvasive  Assessment  of  the  Visual  System  Technical  Digest. 
1992  (Optical  Society  of  America,  Washington,  D.C.)  Vol.  1,  64-67.  ~ 

13-  Klein,  S.  A.  &  Tyler,  C.  W.  (1992).  The  psychophysics  of  visual  detection:  A  review  of  Graham's 
"Visual  Pattern  Analyzers".  (In  press,  Journal  of  Mathematical  Psychology ). 

14.  Klein,  S.  A.  (1992).  Image  quality  and  image  compression:  A  psychophysicist’s  viewpoint.  (In 
press,  chapter  in  Visual  Factors  in  Electronic  Image  Communications.  MIT  Press). 

15.  Klein,  S.  A.  (1992).  Spatial  vision  models:  Problems  and  successes.  (In  press,  chapter  in  Spatial 

Vision  in  Humans  and  Robots.  Cambridge  University  Press’).  ' 

16.  Klein,  S.  A.  (1992).  Will  robots  see?  (In  press,  chapter  in  Spatial  Vision  in  Humans  and  Robots. 
Cambridge  University  Press). 

17.  Waugh,  S.  J.,  Levi,  D.  M.  &  Carney,  T.  (1992).  Orientation,  masking  and  vernier  acuity  for  line 
targets.  (Vision  Research,  In  Press). 

18.  Klein,  S.  A.,  Silverstein,  D.A.  &  Carney,  T.  (1992).  Relevance  of  human  vision  to  JPEG-DCT 
compression.  (In  press,  Human  Vision,  Visual  Processing  and  Digital  Display  III.  Bernice  E. 
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Articles:  Submitted. 

19.  Hu,  Q.,  Klein,  S.  A.  &  Carney,  T.  (1992)  Can  sinusoidal  Vernier  acuity  be  predicted  by  contrast 
discrimination?  Almost.  (Submitted  to  Vision  Research). 
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20.  Carney,  T.  &  Shadlen,  M.  (1992).  Dichoptic  activation  of  the  early  motion  system.  (Submitted  to 
Vision  Research.) 

21.  Klein,  S.  A.  (1992).  Inverting  a  Laplacian  topography  map.  (Submitted  to  Brain  Topography). 

Articles:  In  Preparation.  These  are  articles  that  we  plan  to  complete  before  we  submit 
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22.  Carney,  T.,  Silverstein,  D.  A.  &  Klein,  S.  A.  (1993).  Vernier  acuity  during  image  rotation  and 
translation:  Visual  performance  limits.  (To  be  submitted  to  Vision  Research). 

23.  Silverstein,  D.  A.  &  Klein,  S.  A.  (1993).  Discrimination  of  relative  motion.  (To  be  submitted  to 
Vision  Research). 

24.  Beard,  T.,  Klein,  S.  A.  &  Carney,  T.  (1993).  The  detection  and  discrimination  of  flicker  and  jitter  in 
human  vision.  (To  be  submitted  to  Vision  Research). 
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